[In-Depth Look] A New Way to Interact with Your Phone: Bixby, the Galaxy S8 Intelligent Interface

As the capabilities of smartphones become more diverse, their interfaces are also becoming more complicated, making it difficult for users to take advantage of their features and functions. The Galaxy S8 and S8+ aim to overcome this challenge with Bixby, Samsung’s new intelligent interface that makes interacting with one’s device, services and apps more instinctive and seamless.

Bixby was created to provide a more personalized experience to enhance the way people interact with their phones and the world around them. Bixby is not an ordinary voice assistant, but rather an intuitive and comprehensive interface that leverages contextual awareness to learn users’ habits and respond accordingly.

Bixby offers four main features: Voice, Vision, Reminder and Home.

Voice: A New Way to Interact

Bixby Voice is easily activated. Users simply need to press and hold the dedicated hardware button on the side of the Galaxy S8, say “Bixby” or tap it on the Bixby Home screen to wake it up.

While most traditional smartphone-related tasks require touch activation, Bixby’s multi-modality lets users control their phone using voice and touch controls interchangeably for maximized convenience. Utilizing natural language understanding, Bixby has the ability to adapt to the unique speaking style of the user. But if Bixby doesn’t understand the user’s command, it will ask for more information so it can complete the task, rather than giving up.

Bixby brings a deeper degree of voice integration, but can also easily switch back and forth between each mode of interaction – from voice command to touch, text and back to voice.

Bixby works simultaneously across multiple apps and has the potential to control all settings and features of each service. So, for example, if a user wants to send pictures to a friend, they can select images in their Gallery by tapping them, then tell Bixby to send the selected photos to their designated contact. Meanwhile, Bixby continues to work in the background to continually support the user, without affecting other tasks they might be working on.

Vision: Information in a Snap

With its Vision interface, Bixby can detect what users are looking at and take action according to what they see. Using the Galaxy S8’s Camera app, Bixby makes it easier to search for images and information, shop online and even translate text.

A tourist in Paris, for instance, spots an interesting looking building and snaps a photo of it. Bixby uses the image to determine that the structure is a famous art museum and can then provide related information, such as the museum’s address, phone number and website. At lunch, the user can also leverage Vision to get a translation of the French menu in their own native language or learn more information about the bottle of wine they would like to order.

Shopping is just as easy. Users can take a photo of an item such as a watch or a book and use Vision to locate and purchase it online.* Users can also launch Vision for more in-depth searching of products they see in their phone’s Gallery or while browsing Samsung Internet. Vision can be activated to read QR codes and, in select markets, barcodes that direct users to related information.

Reminder: Friendly Alerts When You Need Them Most

Bixby also helps users remember important events and to-do tasks through its intelligent Reminder feature.

It does so by sending notifications to users reminding them about tasks at specified times or locations, or in certain circumstances. Say a user is out of milk. They can request that Bixby remind them to buy more once they get off work. Or, they can ask Bixby to set their alarm each night before they go to sleep.

When reading an article or watching a video, users can also use Reminder to mark their place should they need to stop and resume where they left off at a later time.

Users can access stored reminders via the Reminder app, Reminder widget, Reminder cards on the Bixby Home screen or the Galaxy S8’s edge screen.

Home: All Important Content in One Convenient Place

Bixby’s Home screen lets users see the content they care most about with just a single swipe on the home screen or a short press of the designated Bixby button.

Home integrates with select native Samsung apps to provide a seamless look at all of the activity across the Galaxy S8 through a customizable card-like interface. For example, Home’s contact card shows frequent and missed call contacts, while music-related cards provide options such as playing paused content. Facebook and other social media cards, meanwhile, display trending content and news cards announce the latest headlines.

But Home doesn’t just relay information; it puts it into context, paying attention to the user’s schedule, routine and personality each time it’s used. In this way, Bixby can actually anticipate what the user needs and respond accordingly.

Say someone often uses Uber to commute to work. Bixby will start to place an Uber card at the top of Home in the morning, with a fare estimate and request button. When it’s time for bed, Home might display a user’s alarm and health summary of daily activity. The more users access Bixby, the more it evolves and adapts.

With its intuitive, comprehensive features and ability to leverage contextual awareness, Bixby provides a more convenient, personalized mobile experience, changing the way people interact with their phones. As the Bixby ecosystem continues to grow, Samsung believes it will evolve from an interface for your smartphone to an interface for your life.

Listen to Injong Rhee discuss Bixby in depth in the interview below.

*At launch, Vision’s shopping service will be available in 11 countries, including the US, UK and Korea, and will expand to others in the future.